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@ Image sensing apparatus. 

@ In an nnage sensing apparatus of this embo- 
diment, a variable angle prism for preventing a 
camera shake Is used to attain a pixel shift 
Consequentfy, low resolution images free from 
a blur and Ngh-resoiution inrages obtained by 
the pixel shift can be selectively obtained witii 
an extremely simple anangement Whether a 
resohjtkMi increasing function is to be executed 
is propeily determined in accordance with 
photographing conditions. Therefore, tiie resol- 
ution increasing function has no adverse effect 
on other functions. In addition, a power-having 
effect can be enhanced since an unnecessary 
resolution increasir^ function is not performed. 
Since the resolution can be further increased by 
using a conventional camera shake preventing 
device, a larger anxujnt of a high-frequency 
component can be detected. Consequentiy, 
focus detection can be performed accurately. 
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@ Image sensing apparatus. 

@ In an Image sensing apparatus of this embo- 
diment a variable angle prism for preventing a 
camera shake is used to attain a pixel shift 
Consequently, low resolution images free from 
a blur and high-resolution images obtained by 
the pixel shift can be selectively obtained with 
an extremely simple anangement. Whether a 
resolution increasing function is to be executed 
is properly detennined in accordance with 
photographing conditions. Therefore, the resol- 
ution increasing function has no adverse effect 
on other functions. In addition, a power-saving 
effiect can be enhanced since an unnecessary 
resolution increasing function is not performed. 
Since the resolution can be further increased by 
using a conventional camera shake preventing 
device, a larger amount of a high-frequency 
component can be detected. Consequently, 
focus detection can be perfbrmed accurately. 
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BACKGROUND OF THE INVENTION 

The present Invention relates to an image sens- 
ing apparatus and, more particularly, to a high- 
resolution image sensing apparatus using an image 5 
sensor such as a charge coupled device (CCD). 

A recent advance of Image sensors such as a 
charge coupled device (CCD) has achieved a high im- 
age quality, a low price, and a small size of image 
sensing apparatuses, e.g., household compact video io 
cameras and still video cameras. Consequently, a 
satisfactory Image quality can be obtained by a tele- 
vision standard of. e.g.. NTSC. In addition, camera 
shake correction is currently performed in video cam- 
eras and the like, and this further improves the image is 
quality of dynamic images. 

A camera shake is a phenomenon in which pho- 
tographed images move vertically or horizontally 
while a user Is perfonfning photographing by holding 
a vkjeo camera in his or her hands, since the hands 20 
or the body of the user slightly moves independently 
of the user's intention. Images thus photographed 
give a viewer a considerable uncomfortableness 
when reproduced on a television monitor or the like. 

To avoid this camera shake phenomenon, con- 25 
ventional video cameras make use of. e.g.. a variable 
angle prism (to be referred to as a "VAP* hereinafter). 

A practteal example of an arrangement of a con- 
ventk>nal image sensing apparatus including a VAP 
for camera shake correction will be described below 3o 
with reference to Fig. 15. 

In Fig. 15, a VAP 104 is constituted by coupling 
two glass plates 104a and 104b via a bellows-Hlike 
spring member 104c and sealing an optically trans- 
parent \k\u\d 1 04d in the space surrounded by the two 35 
glass plates 104a and 104b and the spring member 
104c. Shafts 104e and 104f provided in the glass 
plates 104a and 104b are connected to an actuator 
1 03 for horizontal driving and an actuator 1 06 for vert- 
ical cfriving, respectively. Therefore, the glass plate 40 
104a is rotated horizontally, and the glass plate 104b 
is rotated vertically. 

Note that the VAP 104 is described In Japanese 
Patent Laid-open No. 2-12518 and so a detailed de- 
scription thereof will be omitted. 4S 

A horizontal angular velocity sensor 101 detects 
an angular velocity caused by a horizontal motion of 
the image sensing apparatus resulting from a camera 
shake or the like. A control unit 102 performs an ar- 
ithmetic operation for the detection signal from the so 
angular velocity sensor 101 such that this horizontal 
motton of the image sensing apparatus is corrected, 
and detects and supplies an acceleratton component 
to the actuator 103. This actuator 103 drives the glass 
plate 1 04a of the VAP 1 04 horizontally. ss 

The rotational angle of the glass plate 1 04a which 
can be horizontally rotated by the actuator 103 is de- 
tected by an angle sensor 105. The control unit 102 



perfornr^ an arithmetic operation for this detected ro- 
tatbnai angle and supplies the result to the actuator 
103. 

A vertical angular velocity sensor 1 06 detects an 
angular velocity caused by a vertical motion of the im- 
age sensing apparatus resulting from a camera 
shake or the like. A control unit 107 perfonDs an ar- 
ithmetic operation for the detectton signal from the 
angular velocity sensor 106 such that this vertical mo- 
tk)n of the image sensing apparatus is corrected, and 
detects and supplies an acceleration component to 
the actuator 108. This actuator 108 drives the glass 
plate 104b of the VAP 104 vertically. 

The rotational angle of the glass plate 104b which 
can be vertically rotated by the actuator 1 08 is detect- 
ed by an angle sensor 109. The control unit 107 per- 
fbnfns an arithmetto operation for this detected rota- 
ttonal angle and supplies the result to the actuator 
108. 

An Image sensing optical system 110 forms an 
Image of an object to be photographed on an Image 
sensor 111. This image sensor 111 is constituted by, 
e.g., a CCD. A two dimenstonal sold state CCD is 
used in conventtonal inr^ge sensing apparatuses 
such as vMeo cameras. An output from the image 
sensor 111 Is output to a recording apparatus or a tel- 
eviston monitor through a signal processing circuit 
(not shown). 

In the conventtonal image sensing apparatus 
with the above arrangement, the horizontal and vert- 
k:al angular velocities caused by a camera shake are 
detected. On the basis of the angular velocities de- 
tected, the actuators nwve the VAP horizontally and 
vertk^ally to refract incident light, thereby performing 
control such that the Image of an object to be photo- 
graphed does not move on the Image sensing plane 
of the image sensor. Consequently, the camera 
shake is conrected. 

On the other hand, with recent spread of Image 
sensing apparatuses, an increasing demand has aris- 
en for a higher image quality. To meet this demand. 
It is being attempted to increase the number of pixels 
of an Image sensor such as a CCD In an Image sens- 
ing apparatus or to develop a high-speed I/O appara- 
tus or an Image infbnnatton compressing/decoding 
apparatus. 

For example, the number of pixels of a oonven- 
ttonal two-dimensional solkJ state CCD Is normally 
400.000, and the number of pixels of even a high- 
resolution CCD is at most 2,000,000. These pixel 
numbers currently available are unsatisfactory to ob- 
tain a sufficient resolution necessary for images, 
hard copies, and computer graphks to be displayed 
on large screens. 

The following methods, therefore, are disclosed 
as a method of realizing a high Image quality with a 
limited number of pixels of an image sensor. 

(1 ) A method in which an Image of an object to be 
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photographed is divided by a prism and photo- 
graphed by a plurality of Image sensors, as found 
In some presently aivallable video camera recor- 
ders. 

(2) A method as proposed In, e.g., Japanese Pa- 5 
tent Laid-open No, 60-250789, in which an im- 
age region of a photographing optical system is 
split by, e.g.. a secondary imaging optical sys- 
tem, and the individual split regions are photo- 
graphed by a plurality of image sensors and then io 
synthesized. 

(3) A method of pbcel shift proposed in, e.g., Jap- 
anese Patent Publication Nos. 50-13052, 59- 
18909, and 59-43035. In this method, an elenfient 

for splitting a light beam, such as a dichroic mirror is 
or a half mirror. Is arranged on the Image plane 
side of an image sensing optical system, and a 
plurality of area sensors are arranged to be shift- 
ed from each other by an amount corresponding 
to a half pbcel pitch or smaller. In this method it Is 20 
possible to obtain, from a plurality of resulting Im- 
ages, information containing pbcels In a number 
larger than the numt)6r of pixels of the area sen- 
sors. Also, "Highvision 2/3-inch compact CCD 
camera using dual green method" in Television 2S 
Society Magazine, 'Image Information Engineer- 
ing and Broadcasting Technology", Vol. 47, No. 2, 
1993 has reported a niethod in which two of three 
CCDs for receiving separated light beams from a . 
trichromatic separation prism are arranged such 30 
that their pixels are shifted from each other. In 
this method an object to be photographed is pho- 
tographed with this pbcel-shifted arrangement 

(4) Japanese Patent Publication No. 57-31701 or 
64-863 has disclosed a method by which a pbcel 35 
shift is performed by nx)ving the position of inci- 
dent light to an image sensor relative to the image 
sensor In synchronism with the read timing df an 
outptJt electrical signal from the sensor. This Is 
accomplished by, e.g., Inserting a birefringence 40 
polarizing element into a photographing optical 
system and controlling the element By periodi- 
cally photographing the resulting images, it is 
possible to obtain information containing pbcels in 

a number larger than the number of pbcels of the 45 
image sensor. Research and development of this 
pixel shifting method have been made extensh^e- 
ly to date. 

As discussed above, the camera shake prevent- 
ing function and the resolution increasing function so 
have been developed in conventional image sensing 
apparatuses. Unfortunately, in these conventional 
image sensing apparatuses the camera shake pre- 
venting function and the resolution increasing func- 
tion are independent of each other. Therefore, the 55 
conventtonal image sensing apparatuses must have 
independent arrangements to include both of the two 
functions. An image sensing apparatus including 



both these functions, therefore, is difficult to manu- 
facture at a low cost and miniaturize. 

In addition, in a conventional image sensing ap- 
paratus which realizes a high-resolution by using the 
pixel shifting method as described above, this pbcel 
shifting function and various other automatic adjust- 
ing functions of the image sensing apparatus may in- 
terfere with each other, thereby causing a function 
failure. As an example, if a pbcel shift is performed by 
slightly vibrating a focusing lens group or an image 
sensor in the optical axis directton while an automatic 
focusing device is operating using TV signal AF (Auto 
Focus), no stable peak value output of a luminance 
signal can be obtained. This degrades the original 
performance of the automatk: focusing device. Also, 
if a pixel shift is done while another automatic adjust- 
ing device is In operation, out-of-fbcus images or over 
exposed-images result in some cases. In addition, if 
a pbcel shift is performed during zooming, the moving 
anKXjnt of the position of incident light to an image 
sensor cannot be held constant Consequently, it may 
become impossible to obtain a satisfactory image 
quality. 

Furthennore, since a pbcel shift is done although 
no satisfactory effect can be expected, the resoiutton 
Increasing function operates unnecessarily, consunv 
Ing an extra power. This is espedally a sertous prob- 
lem in portable image sensing apparatuses in which 
power is supplied by a battery. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide 
an image sensing apparatus including both of a cam- 
era shake preventing function and a resolutton in- 
creasing f unctk>n with a simple arrangement 

According to the present invention, the foregoing 
object is attained by providing an intage sensing ap- 
paratus comprising image forming means for forming 
an Image of an object to be photographed. Image 
sensing means for photographing the image of the 
object fonned by the Image forming means, displao- 
ing means for displacing the image of the object 
formed by the image forming means and the image 
sensing means relative to each other, and control 
means for performing switching between a first inv 
age sensing mode in which the displacing means per- 
forms a vibration conrectton to yield a low resdutkm 
image and a second image sensing mode in which 
the displacing means performs a pixel shift to yield a 
high-resolution image. 

In accordance with the present invention as de- 
scribed above, the first or second image sensing 
mode is selected either arbitrarily t>y a user or auto- 
matically. Consequently, when dynamic in>ages are 
required, information of dynamic images free from a 
blur can be obtained by moving the relative positions 
of an image of an object to be photographed, which 
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is formed by an optical system, and an image sensor 
by an amount by which a camera shake is corrected. 
When a high-resolution Is necessary, the relative 
positions of the image of the object formed by the opt- 
ical system and the image sensor are finely moved to s 
photograph a plurality of images, and these images 
are synthesized. 

It is another object of the present invention to pro- 
vide an Image sensing apparatus capable of properly 
determining whether a resolution increasing function 10 
is to be executed, thereby enhancing a power-saving 
effect 

According to the present invention, the foregoing 
object Is achieved by providing an Image sensing ap- 
paratus comprising image forming means for forming is 
an image of an object to be photographed, Image 
sensing means for photographing the image of the 
object formed by the inrmge forming means, displac- 
ing means for displacing the image of the object 
formed by the image forming means and the image 20 
sensing means lelative to each other, synthesizing 
means for obtaining a high-resolution image by syn- 
thesizing a plurality of images of the object obtained 
by performing a pbcel shift by the displacing means, 
changing means for changing photographing condl- 25 
tions, and control means for determining, in accor- 
dance with the photographing conditions changed by 
the changing means, whether the displacing means is 
to be driven. 

in accordance writh the present invention as de- 30 
scribed above, driving of the displacing means for 
performing a pbcel shift can be controlled in accor- 
dance with the photographing conditions. Conse- 
quently, an unnecessary pixel shift is not perfonned. 

It is another object of the present Invention to pro- 35 
vide an image sensing apparatus capable of further 
increasing the resolution by using an image sensor 
with a limited number of pUels. 

According to the present Invention, the foregoing 
object Is attained by providing an image sensing ap- 40 
paratus comprising image forming means for forming 
an image of an object to be photographed, image 
sensing means for photographing the image of the 
object formed by the image forming means, displac- 
ing means for displacing the Image of the object 45 
formed by the image forming means and the image 
sensing means relative to each other, synthesizing 
means for obtaining a high-resolutton image by syn- 
thesizing a plurality of Images of the object obtained 
by performing a pixel shift by the displacing means, so 
and vibration correcting means for selectively per- 
forming a vibration correction by using the displacing 
means. 

In accordance with the present invention as de- 
scribed above, the resolution can be further In- ss 
creased in addition to being able to perform a vibra- 
tion correction. 

The Invention is particularly advantageous since 



there can be provided an image sensing apparatus by 
which an image recording mode is selected either ar- 
bitrarily by a user or automatically, and so It is possi- 
ble to selectively obtain low resolution images free 
from a blur caused by a vibration such as a camera 
shake and high-resolution images. Since these two 
types of images are formed by using a common ar- 
rangement, the configuration of the Image sensing 
apparatus is exceedingly simplified. 

In addition, whether the resolution increasing 
function is to be executed is properly determined. 
Consequently the power-saving effect is enhanced, 
and the resolution increasing function is prevented 
from having an adverse influence on other operations 
of the apparatus. 

Furthermore, since a high-resolution can be real- 
ized by using a conventional camera shake prevent- 
ing device, a larger amount of a high-frequency com- 
ponent can be detected. This makes an accurate fo- 
cus detectk>n feasible. 

Other features and advantages of the present in- 
ventk>n will be apparent from the following descri(>- 
tbn taken in conjunctk)n with the accompanying 
drawings, in which like reference characters desig- 
nate the same or similar parts throughout the figures 
thereof. 

BWEF DESCRIPTION OF THE DRAWINGS 

The accompanying drawings, which are incorpo- 
rated In and constitute a part of the specification, il- 
lustrate embodiments of the invention and, together 
with the description, serve to explain the principles 
the invention. 

Fig. 1 is a btock diagram showing the arrange- 
ment of an image sensing apparatus according to 
the first embodiment of the present invention; 
Figs. 2A and 2B are views showing the relatkm- 
ship between a pbcel arrangement and an image, 
and the pbcel arrangement of a high-resolution 
image after synthesis, respectively, in a second 
image sensing mode of the first embodiment; 
Fig. 3 is a btock diagram showing the arrange- 
ment of an Image sensing apparatus according to 
the second embodiment of the present inventfon; 
Figs. 4A and 4B are views showing the reiatton- 
shlp between a pbcel anrangement and an image, 
and the pbcel arrangement of a high-resolutton 
image after synthesis, respectively, in a second 
image sensing mode of the second embodiment; 
Figs. 5A and 5B are block diagrams showing the 
anrangement of an image sensing apparatus ac- 
cording to the third embodiment of the present in- 
vention; 

Figs. 6A to 60 are views for explaining focus de- 
tection according to TV signal AF in the third em- 
bodiment; 

Fig. 7 is a flow chart showing pixel shift start de- 
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termination processing in the third emt)odiment; 
Fig. 8 is a t}lock diagram showing the arrange- 
ment of an image sensing apparatus according to 
the fourth emt>odlment of the present invention; 
Fig. 9 is a timing chart of CCD driving in the fourth 
emtxxjiment; 

Fig. 10 is a view showing the way the position of 
an image of an object to be photographed is shift- 
ed on an image sensor in the fourth embodinrent; 
Fig. 11 is a view showing the relationship be- 
tween the Inclined angle of a light beam and the 
shifted amount of an image of an object in the 
fourth embodiment; 

Fig. 12 Is a view showing the way an image is 
shifted In the fifth embodiment of the present in- 
vention; 

Fig. 13 is a \Aock diagram showing the arrange- 
ment of the sixth embodiment of the present in- 
vention; 

Figs. 14A to 14E are views showing other means 
for performing a pbcel shift in the first to sbcth em- 
bodiments; and 

Fig. 1 5 is a Mode diagram showing an iiiustrative 
arrangement of a conventional image sensing 
apparatus. 

DESCRIPTION OF THE PREFERRED 
EMBODIMEI^ 

Preferred embodiments of the present invention 
will now be described in detail in accordance with the 
accompanying drawings. 

<1st Embodiment> 

Fig. 1 is a block diagram showing the basic ar- 
rangement of an image sensing apparatus according 
to the first embodiment of the present invention. In 
Fig. 1, the same reference numerals as In Fig. 15 
which illustrates one conventional example denote 
the same parts, and a detailed description thereof wilt 
be omitted. 

Referring to Fig. 1, reference numeral 151 de- 
notes a pbcel number controller 151. In an image sen- 
sor 111, light-receh^ing portions of Indh/idual pixels 
are arranged In two directions, the X and Y directions, 
with a spacing larger than the light-receiving portion 
of one pbcel. in this embodiment, the relath^e positions 
of an image of an object to be photographed, which 
is formed by an image sensing optical system 110, 
and the image sensor 111 are displaced, and the pos- 
ition of a VAP 104 Is changed such that the light- 
receiving portions of the individual pbcels of the image 
sensor 111 apparently cover a light-insensitive por- 
tion between the light-receiving portions. The pixel 
number controller 1 51 outputs a control signal for this 
purpose to control units 102 and 107. On the basis of 
the output control signal from the pixel number con- 



troller 151, the VAP 104 is driven by actuators 103 
and 108. 

To use the Image sensing apparatus of this em- 
bodiment a user first detemilnes the operation 

5 mode. Reference numerals 152 and 153 are mode 
switches for this operation. The Image sensing appa- 
ratus of this embodiment has a first image sensing 
mode in which camera shalce prevention is executed 
although the resolution is relatlv^y low, and a second 

10 Image sensing mode in which the resolution is rela- 
tively high but no camera shalce prevention is per- 
formed. Each of the mode switches 152 and 153 is 
closed to a position a in the f irst image sensing mode 
and to a position b in the second image sensing 

16 mode. In the first image sensing nnxie, therefore, the 
VAP 1 04 is so controlled as to correct a blur caused 
by a vibration such as a camera shake on the basis 
of horizontal and vertical angular M^odiy Information 
(acceleration informatk>n). In the second image sens- 

20 ing mode, on the other hand, the VAP 1 04 Is so con- 
trolled that the number of pbcels Is seemingly in- 
creased, i.e.. the light-receiving portions of the pixels 
apparently cover the light-insensitive region between 
the light-receiving portions. 

25 An analog signal obtained by the Image sensor 
1 1 1 is converted into a digital signal by an AJD conver- 
ter 154 and applied to a scan converting unit 155 in- 
cluding frame memories 1 56a to 1 56d. In the first im- 
age sensing nxKle. the scan converting unit 155 does 

30 not f unctk>n. That is, the digital Image signal is always 
written in, e.g.. the frame memory 1 56a and output in- 
tact via a data selector 157. In the second image 
sensing nrK>de, on the other hand, the scan converting 
unit 155 stores the output digital image signal from 

35 the A/D converter 1 54 divisionally in frame memories 
1 06a to 1 06d using a method to be described in detail 
later. The data selector 157 of the scan converting 
unit 1 55 outputs a high-resolutk>n image by alternate- 
ly reading the frame memories 1 56a to 1 56d. The out- 

40 put signal In whtoh the number of pbcels Is Increased 
by the scan converting unit 155 Is output to a record- 
ing unit 159, a monitor display 160, or a printer 161 
through a signal processing unit 158. 

Details of the second image sensing nKXie for ot>- 

45 taining high-resolutk)n images will be described be- 
low. Figs. 2A and 2B are plan views each showing, in 
a time series manner, a relative nMvement between 
the tight-receiving portion of the image sensor 111 
and an image of an object to be photographed in the 

50 image sensing apparatus of this embodiment Note 
that in Figs. 2A and 2B, it is assumed that an image 
of an object to be photographed is f bced in position. 
Fig. 2A is a view for explaining 0ie pbcel anrangement 
in the light-receiving portion of the image sensor 111. 

55 In Fig. 2A, the light-receiving portk3n of the image 
sensor 111 consists of NxM pixels, i.e., N pixels in the 
X direction and M pixels in the Ydlrectnn. For simpli- 
city of explanation, some of these pbcels are extracted 

5 
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in Fig. 2A. Of the extracted pixels, a pixel of interest 
is given a pixel number, 'r, and its position is indicat- 
ed by coordinates (1,]). Fig. 2B shows the pixel ar- 
rangement obtained by synthesis when the data se- 
lector 1 57 alternately reads the f ranne memories 1 56a s 
to156d. 

Referring to Fig. 2A, In the light-receiving portion 
the pixel of interest is moved relatively in order of "1" 
"2" "3" -> "4'* for each frame of the Image sensor 
111 by the VAP 104 which is driven by the actuators io 
103 and 108 on the basis of the control signal from 
the pixel number controller 151. That is, the pixel of 
Interest is shifted by 1/2 of the distance between in- 
dividua) light-receiving portions. Assuming the 
frames coirespondlng to the pbcel positions "1 " to "4" is 
in Fig. 2A are first to fourth frames, the frame period 
of an image synthesized by the data selector 157 is 
given by the sum of the first to fourth frame periods. 

In each of the first to fourth frame periods, there- 
fore, the image data from the image sensor 111 are 20 
sequentially stored in the corresponding frame menv 
ory. The data thus stored are read out by the data se- 
lector 157 and finally recorded, displayed, or printed. 
An image obtained as the final output at this time is 
synthesized as illustrated in Rg. 2B. Referring to Fig. 25 
2B, the total number of pixels is 2Nx2M, and pixels 
photographed during the individual corresponding 
frame periods by the positions "1' to "4" of the pixel 
of interest. Fig. 2A, are present at positions (21-1 ,2i- 
j), (2i,2j-1), (2i-1 ,2j), and (2i,2j). That is. the resolution 30 
is doubled in both the X and Y directions. 

The above explanation is made assuming that an 
Image of an object to be photographed is f bced In pos- 
ition. In practice, since an image of an object is re- 
fracted by the VAP 104, the relative position of the inv 35 
age with respect to the inrtage sensor 11 1 is displaced. 

As discussed above, the image sensing appara- 
tus according to this embodiment includes the first 
image sensing mode in which camera shake preven- 
tion is performed although the resolution is low, and 40 
the second image sensing mode in which the resolu- 
tion is high but no camera shake preventton Is done. 
Consequently, a user can selectively obtain and out- 
put high-resolutton images and images free from a 
blur. 45 

It is also possible to output high-resolution inrv 
ages thus photographed as a dynamic image by re- 
peatedly performing photographing in the second im- 
age sensing mode described above. FurthenTX>re, 
during photographing in the first image sensing so 
mode, the operation nrKXle can be switched to the sec- 
ond image sensing fx\ode only for one cycle from the 
first to fourth frames mentioned above. This makes it 
possible to obtain a still image with a desired high-r 
resolution. 55 



<2nd Embodiment> 

The second embodiment of the present inventfon 
will be described below. 

Fig. 3 is a block diagram showing the arrange- 
ment of an image sensing apparatus according to the 
second embodiment of the present inventkin. In Fig. 
3, the same reference numerals as in Fig. 15 which 
illustrate one conventional example denote the same 
parts, and a detailed descriptton thereof will be omit- 
ted. 

Referring to Fig. 3, reference numeral 162 de- 
notes a motk)n detector for detecting the motk>n of an 
object to be photographed from a signal obtained by 
A/D converting the output from an image sensor 111. 
A motion detectfon method in this second embodi- 
ment can be, e.g., the n^ethod described In Japanese 
Patent Laki-Open No. 1-98563. in this method, a plur- 
ality of images photographed at different times 
(fields) are compared in units of fine bfocks. thereby 
detecting the motion vector of the object It is of 
course possible to employ other well-known methods 
of mcAion detectnn. 

Image sensing processing in the second embodi- 
ment having the above arrangement will be described 
below. As with the image sensing apparatus of the 
first embodiment discussed above, the image sens- 
ing apparatus of the second embodiment has a first 
Image sensing mode for preventing a camera shake 
and a second image sensing mode for obtaining high- 
resolution images. To photograph an object to be pho- 
tographed by using the image sensing apparatus of 
the second embodiment, mode switches 1 52 and 1 53 
are first cfosed to their respective positfons a to per- 
form photographing in the first image sensing mode. 
If the motion detector 162 determines that the object 
is moving, the image sensing apparatus immediately 
performs recording, display, or the like in the first im- 
age sensing nrx>de as it is. 

On the other hand, if the motion detector 162 de- 
termines that the object is at rest, the mode switches 
152 and 153 are dosed to their respective positfons 
b, thereby autonrntteally switching the mode of the Inn- 
age sensing apparatus to the second image sensing 
mode for obtaining a high-resolutk>n. 

As described above. In the second embodinrtent 
the photographing mode is automattealiy svvitched 
between the first and second nrKxjes In accordance 
with the moving amount of an object to be photo- 
graphed. This makes it possible to perform photo- 
graphing in the f tret image sensing nKxJe If an object 
is moving, and in the second image sensing nK>de if 
an object is at rest Consequentfy, optimum images 
can be obtained constantly. 

In the above firet and second embodiments, in 
the hlgh-resdutfon second image sensing mode the 
number of pixels is increased at the same magnifica- 
tion in the X and Y directions of an image of an object 
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to be photographed. It is of course possiUe to in- 
crease the number of pbcels at different nnagnifica- 
tions in the X and Y directions. It is also possible to 
shift an image of an object and the in^age sensor 111 
relative to each other only In, e.g., the X direction. s 

In addition, in the above embodiments an image 
of an object to be photographed is displaced such that 
the Individual light-receiving portbns of the image 
sensor 111 apparently cover the light-insensitive re- 
gion between the light-receiving portions. An output io 
received during the displacement of each light- 
receiving portion is displayed or recorded as an out- 
put from one pixel. This is equivalent to improving the 
resolution in both the X and Y directions. However, 
the present invention is not limited to the above env is 
bodiments. For example, the number of pixels can be 
increased by displacing an image of an object to be 
photographed such that the light-reoelving portions 
apparently overlap each other. 

Figs. 4A and 4B are plan views each showing, in zo 
a tinf\e series manner, a relative moven^ent between 
the light-receiving portions of the image sensor 111 
and an image of an object to t>e photographed. Note 
that in Figs. 4A and 4B, it Is assumed that an image 
of an object to be photographed Is f bced, as In Figs. 25 
2A and 2B. In Fig. 4A, the light-receiving portions of 
the image sensor 1 1 1 are moved to seemingly overlap 
each other. Consequently, like in Figs. 2Aand 28 dis- 
cussed earlier, an Image with a doubled resolution in 
both the X and Y directions can be obtained as lllu- 30 
strated in Fig. 48. In this case, if each overlapped por- 
tion between the light-receiving portions is 1/2 of the 
light-receiving portion, MTF (Modulation Transfer 
Function) is weakened. Consequently an aliasing dis- 
tortion caused by the image sensor 111 is sup- $5 
pressed. This eliminates the need for an optical low 
pass filter consisting of a quartz crystal or the like, 
which is usually attached to the front surface of the 
Image sensor 1 11 in order to remove any aliasing dis- 
tortk>n. 40 

Note that in the above first and second embodi- 
ments, in the second image sensing mode the rela- 
tive posittons of an image of an object and the Image 
sensor 111 are not changed in accordance with vibra- 
tion detection. Sincethefrequency of a camera shake 45 
Is generally about 10 Hz, a control signal for vibration 
conrection can be superposed on a control signal for 
Increasing the number of pixels which Is supplied by 
the pixel number controller 151. As a result, high- 
resolution images free from a blur can be obtained in so 
the second image sensing mode. 

Note also that in the first and second embodi- 
ments discussed above, the relative positions of an 
image of an object to t>e photographed and the light- 
receiving portions of the image sensor 1 1 1 are shifted ss 
by a 1/2 pbcel pitch in the horizontal drectton and/or 
the vertical directton. This pitch need only be 1/N (N 
is a natural numt)er of 2 or larger) of the pixel pitch. 



In this case, N images of an object, of course, must 
be photographed. 

<3rd Embodiment> 

The third embodiment of the present inventk>n 
will be described below. 

In this third embodiment, an image sensing appa- 
ratus including a zoom functkm will be described. 
Figs. SAand 58 show the arrangement of the image 
sensing apparatus according to the third embodh 
ment Reference numerals 201 a to 201 d denote zoom 
lens groups. In which 201 b denotes a variator lens 
group for varying the nrragnification; and 201d, a fo- 
cusing lens group. In Figs. 5A and 58, these zoom 
lens groups are constituted by lens groups having 
positive, negative, positive, and positive refracting 
powers. However, various arrangements are possible 
as an optical system having a variable magnificatton 
function or a focusing f unctton. Therefore, the pres- 
ent inventton is not limited to the arrangement shown 
in Figs. 5A and 58. 

Reference numeral 202 denotes an aperture 
stop; and 203, an interline type CCD. The quantity of 
light entering the zoom lens groups 201 is adjusted by 
the aperture stop 202, and an Image of the light is 
formed on the CCD 203. A VAP 204 refracts the light 
entering the zoom lens groups 201 and thereby can 
move the image formation position of the incklent 
light on the CCD 203. 

Actuators 205, 206, 207, and 208 drive their re- 
spective corresponding moving parts, i.e., the VAP 
204, the variator lens group 201b. the aperture stop 
202, and the focusing lens group 201d via drivers 
205', 206', 207', and 208*. respectively. Reference 
numerals 209 to 212 denote encoders. The encoder 
209 outputs a vertcal angle 4 of the VAP 204 in the 
form of, e.g., a voltage. The encoders 210 and 212 
output the absolute posittons of the variator lens 
group 201 b and the focusing lens group 201 d, respec- 
tively, by voltages. The encoder 211 outputs the aper- 
ture amount of the aperture stop 202 and hence the 
Fno. (F-number) of the entire optteal system by a vol- 
tege. These output values except the aperture value 
are supplied to a CPU 224 as a lens group control 
means. The aperture value is differentiated by a dif- 
ferentiating circuit 21 3 and supplied to the CPU 224. 

An auto focus (AF) mechanism of the third em- 
bodiment will be described below. Note that in the 
third embodiment, the AF mechanism is described by 
taking so-called TV signal AF as an example, in which 
the focusing lens group Is moved such that the fre- 
quency component of a luminance signal of a video 
signal becomes highest However, the present inven- 
tion can be similarly applied to ohter method, e.g., a 
TCL method or infrared AF. 

An output video signal from the CCD 203 is 
passed through a video signal processor 214 and lim- 
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ited in region by a gate 219. A peak detector 220 de- 
tects a peak value of a iunrtlnance signal in a prede- 
termined regk>n and sends the peak value to the CPU 
224 as informatton So pertaining to the current focus- 
ing state. When an oscillator 223 supplies a driving 5 
signal of a constant period to the driver 208* via the 
CPU 224. the focusing lens group 201 d is so driven 
as to finely oscillate in the optical axis direction. Con- 
sequently, the output So from the peak detector 220 
also oscillates in synchronism with the oscillatton of io 
the lens group. This state Is shown In Fig. 6A. 

In Rg. 6A, the output So from the peak detector 
220 is plotted on the ordinate, and the positbn of the 
focusing lens group 201 d is plotted on the abscissa. 
A positk)n Pm at which the peak value of So is obtained is 
is an in-focu8 positk>n. Therefore, If the focusing lens 
group 201d finely oscillates in an out-of-focus state, 
e.g., when the focusing lens group 201d Is located at 
Pn or Pf:, the phase of the output So with respect to the 
directton of displacement of the focusing lens group 20 
201d Is shifted 180'' In accordance with whether the 
focusing lens group 201 d is positioned on the near 
distance side (Pn side) or on the far distance skle (Pp 
side), as indtoated by amplitudes A^ and Ap of the out- 
put So at their respective poslttons. 25 

Near focus and far focus, therefore, can be deter- 
mined by sending the output from the peak detector 
220 to a phase comparator 222 via a frequency de- 
tector 221 and comparing the output with the phase 
of the reference signal from the osdltator 223. Ampli- 30 
tudes A of the output are A^ and Af when the focusing 
lens group 201d is posittoned on the near focus and 
the far focus skies, respectively, and Am = 0 in an in- 
focus state. 

By performing synchronous detectton for these 35 
signals using the output from the oscillator 223 as a 
reference timing, a signal St, Fig. SB, is obteined. 
That is, assuming the signal in the near distance is in 
phase with the reference timing, a positive signal Is 
output on the near distence side, and a negative sig- 40 
nal is output on the far distance side, as the synchron- 
ous detection signal Si . Note that if the circuit conf ig- 
uratton Is made such that the far distance signal is In 
phase with the reference signal, a negative signal is 
output on the near distance side, and a positive signal 45 
is output on the far distance side, as the synchronous 
detectton signal Si. 

As discussed above, the value of the signal Si is 
"0" in an in-focus state, and its absolute value increas- 
es with the defocus amount i-lowever, the signal Si so 
decreases as the defocus amount increases. There- 
fore, If the signal So becomes smaller than a refer- 
ence level Vh, a signal to be used is not the signal Si 
but a signal S2, Fig. 6C, obtained by comparing the 
signal So generated when the focusing lens group 55 
201 d is located In the Immediately preceding field 
with the signal So generated at the current position of 
the lens group. Since the signal S2 shown in Fig. 6C 



is the comparator output, this signal is equivalent to 
a dtrectton signal which Indicates the ascending por- 
tion or the descending portion of the peak shown in 
Fig. 6A. That is, the signal Indicates the directton in 
which the lens group is to be driven with a given value 
either in the positive or negative direction from the in- 
focus position. More spedficalty, if the defocus 
anfKMjnt is large (So < Vh). the actuator 208 is driven 
at a fixed speed in accordance with the signal S2 (a 
hill-dimbing mode). If the defocus amount is near the 
In-focus position (So ^ Vh), the actuator 208 is driven 
In accordance with the signal Si (a ctosed loop mode). 
Consequently, an in-focus state can be obtained in 
either case. 

Of a series of operations discussed above, the 
fine oscillation of the focusing lens group 201d must 
be done in synchronism with driving of the CCD 203. 
The CPU 224 controls the operaticm such that the tim- 
ing at which the video signal processor 214 receives 
the signal from the CCD 203 and the output timing of 
the oscillator 223 are synchronized with a field pulse 
signal O/E (O: odd field/E: even-f leM) generated by 
a timing pulse generator 229. 

An operation of increasing resolutkui by using a 
pixel shift according to the third embodiment will be 
described bebw. in Figs. 5A and 5B, reference nu- 
noeral 227 denotes a dock driver for driving the CCD 
203. A delay circuit 228 delays transfer pulse signals 
(^1 , ^2) from the timing pulse generator 229 by a 1/2 
pixel in accordance with whether the field Is an even- 
field or an odd field. A switch 231 is used to send field 
information to the delay circuit 228. The swrtoh 231 is 
interlocked with a switch 230 for turning on and off a 
periodical pixel shifting operatton. The switches 230 
and 231 are turned on or off together. In performing 
the pixel shift, these switohes 230 and 231 are turned 
on, and the delay circuit 228 checks whether the f ield 
pulse signal (Q/E) from the timing pulse generator 
229 indicates an even-field or an odd field. If the tino- 
ing pulse signal Is an even-f tekl signal, the delay cir- 
cuit 228 delays the transfer pulse signals (^1 , 4|>2) for 
a read horizontal shift register by a 1/2 pixel. If the 
timing pulse signal Is an odd field signal, the delay cir- 
cuit 228 sends the transfer pulse signals ds'ectly to 
the CCD driver 227 without delaying the signals. Al- 
ternatively, the delay circuit 228 delays the transfer 
pulse signals (^1 , ^2) for the read horizontal shift reg- 
ister by a 1/2 pbcel if the f lekl pulse signal is an odd 
field signal, and does not delay the transfer pulse sig- 
nals if the field pulse signal is an even-field signal. 
The CCD 203 is driven in synchronism with the pulse 
signals (<^1, ^2) from the timing pulse generator 229. 
The VAP 204 is also driven such that the Image for- 
mation position is horizontally shifted a 1/2 pixel on 
the image sensing plane for each field. Consequent- 
ly, It is possible to obtain resolution equivalent to that 
obtained when the number of pucels in the horizontal 
directk>n is nearly doubled. 
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On the other hand, if the switches 230 and 231 
are OFF, the field pulse signal (0/E) is not supplied to 
the delay circuit 228. Therefore, the transfer pulse 
signals (^1, ^2) are constantly supplied to the CCD 
driver 227 after being delayed by a 1/2 pixel, and the 5 
periodical driving of the VAP 204 is stopped. That is, 
ON/OFF of the switches 230 and 231 corresponds to 
ON/OFF of the pbcel shift mode. 

When the switches 230 and 231 are ON, the tim- 
ing the signal read out from the CCD 203 is shifted io 
by a 1/2 pbcel In accordance with whether the field Is 
an odd field or an even-field. Therefore, by outputting 
this signal as a composite signal via a camera encod- 
er 232, on the receiving apparatus side a high- 
resolution image can be obtained on, e.g., a high- is 
resolution nrwnitor from signals of both the fields. 

Note that in Figs. 5A and 5B, a user can visually 
check the output from the camera encoder 232 
through an EVF (Qectric View Finder) 233. The reso- 
lution of the EVF 233 can be relatively low when its 20 
performance Is taken into account However, in the 
high-re8olutk>n nnxle, i.e., when the switches 230 and 
231 are ON, characters such as "HD" are superinrv 
posed on the display screen of the EVF 233. This al- 
lows a user to oonf inn in the low resolution EVF 233 25 
that photographing is currently being done in the hlgh- 
resdutton mode. 

Note that in the third embodiment, the read drive 
timing of the CCD 203 is shifted a 1/2 pbcel in even 
and odd fields. However, it is also possible to write 30 
even-field signals in every other pixels of a memory 
having the numt)er of horizontal pixels which is twice 
the number of pfocels of the CCD 203, and to write odd 
field signals between the even-field signals, in this 
case the signals are sequentially read out pixel-by- 35 
pixel in the horizontal directk>n from the men>ory. Al> 
ternatively, a first field signal can be temporarily stor- 
ed in a menK>ry having the same number of pixels as 
that of the CCD 203. In this case, in reading out a sec- 
ond-field signal the signal from the memory and the 40 
output signal from the CCD 203 are alternately output 
pbcel-by-pixel in the horizontal directk)n. 

Automata exposure control In the third embodi- 
ment will be described below. 

In Figs. 5A and 5B, an optnal image of an object 45 
to be photographed taken by the taking lens groups 
201a to 201d is formed on the image sensing plane 
of the CCD 203 after the light quantity of the Image 
is adjusted by the aperture stop 202. The output video 
signal from the CCD 203 is limited in regbn by a gate so 
215 and subjected to integrating detection performed 
by an integrating detector 216. The resulting signal is 
output to an operational amplifier 21 7 as the intensity 
of luminance. The operational amplifier 217 conv 
pares this voltage value with the voltage of a refer- ss 
ence voltage generator 218, amplifies the resultant 
difference signal, and outputs the signal to the CPU 
224 as a difference signal D2. The CPU 224 supplies 



a signal to the driver 207' such that this difference 
voltage becomes "0", thereby driving the aperture ac- 
tuator 207. In order that the aperture stop 202 is 
opened/dosed stably, the CPU 224 perfonns control 
such that a differential output D^ indicative of the 
aperture position from the differentiating circuit 213 
is fed back and so the ievet of Di is held at a prede- 
termined value. With the above operation, exposure 
Is automatically optimized in the third embodiment 
Therefore, whether the automatic exposure control is 
completed can be determined by checking whether 
the output Di from the differentiating circuit 213 is 
close to "0" or whether the difference signal D2 from 
the operational amplifier 217 is dose to "0". 

A zoom function in the third embodiment will be 
described below. 

Referring to Figs. SAand SB, when a zoom switch 
226 is turned on a directk>n signal corresponding to 
the depressed direction (T side/W side) of the zoom 
switch 226 is supplied to the CPU 224. Note that the - 
zoom switch 226 designates executton of dose up 
photographing in a tele mode when depressed to the 
T side and execution of wkie-angle photographing in 
a wkie mode wtien depressed to the W side. Upon re- 
ceiving the directton signal, the CPU 224 sends a sig- 
nal to the driver 206' to nnove the variator lens group 
201 b in a predetermined directton. The optical system 
used in the third embodiment Is a so-called variable 
focal lens group in which the focusing lens group 201d 
is doser to the image plane than the variator lens 
group 201 b. Consequently, the in-focus plane moves 
with movement of the variator lens group 201b. 
Therefore, the CPU 224 drives the variator lens group 
201 b and at the same time must also drive the focus- 
ing lens group 201d on the basis of data from a ROM 
225 in order to keep the in-focus state of the CCD 203. 
Note that if the optical system is not a variable focal 
lens group, the focusing lens group can be kept f bced. . 
Whether the zoom operation is completed can t>e de- 
termined by checking ON/OFF of the signal from the 
zoom switch 226. 

An AF operation performed by the CPU 224 will 
be described below with rsferenoe to Fig. 7. 

Referring to Fig. 7, in step S10 the CPU 224 
starts the AF operation. Thereafter, the CPU 224 
turns off the switches 230 and 231 In step S11 and 
also turns off the display of characters "HD" indicat- 
ing the high-resolution mode from the image in the 
EVF 233 in step S12. In step SI 3, the CPU 224 
checks whether Sq > Vh as described above in Fig. 
6A. If So ^ Vh. the flow advances to step S14 because 
the defocus anKXjnt is large. In step S14, the CPU 224 
checks whether S2 > 0. If S2 > 0, the CPU 224 moves 
the focusing lens group 201 d toward the far distance 
side in step S15. If S2 ^ 0, the CPU 224 moves the 
focusing lens group 20 Id toward the near distance 
skJe in step SI 6. The CPU 224 repeatedly executes 
this processing until So ^ Vh is obtained in step 813. 
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If So ^ Vh in step S13. the flow advances to step 
S17. and the CPU 224 checks whether IsJ > Vsv 
Vsi is a predetermined value dose to "0". If I Si I > 
Vsi, the CPU 224 so drives the focusing lens group 
201dthat I Si I =: 0 in step SI 8. 

If I Si I ^ Vsi in step S17, the CPU 224 determi- 
nes that an In-focus state is obtained. Therefore, in 
step S19 the CPU 224 stops the focusing lens group 
201 d. 

Thereafter, the flow advances to step S20, and 
the CPU 224 checks whether Di > Vqi is established 
for the differential output Di which is supplied from 
the differentiating circuit 213 and Indicates the aper- 
ture position of the aperture stop 202. Vqi is a prede- 
termined value dose to "0*. If Di > Vqi. in step S21 the 
CPU 224 checks whether D2 > V02 is established for 
the difference signal D2 from the operattonal amplifi- 
er 21 7. V02 is also a prBdeterntined value dose to "0". 
If D2 > V02. in step 322 the CPU 224 perfbnfns the au- 
tomatic exposure control discussed above, and the 
flow advances to step S26. 

If Di ^ Vol in step S20 or D2 ^ V^a in step S21 , 
the flow advances to step S23, and the CPU 224 
checks whether the zoom switch 226 is ON. i.e., is de- 
pressed. If the zoom switch 226 is OFF, the flow ad- 
vances to step S24, and the CPU 224 turns on the 
sviotches 230 and 231. In step S25, the CPU 224 su- 
perimposes the characters "HD" on the EVF 233. This 
display is not limited to "HD" but can be an k;on or 
some other characters. If the zoom switch 226 Is ON 
in step S23, the flow advances to step S26. 

In step S26, the CPU 224 checks whether | Si | 
> Vs2. Vs2 is a predetermined value meeting Vs2 > Vsi • 
If I Si I > Vs2 In step S26, the flow returns to step S11 ; 
otherwise, the flow returns to step S19. Since the re- 
lation V82 > Vsi is established in this manner, no hunt- 
ing occurs in the AF processing even if the level of Si 
is unstable. 

The AF processing is performed as described 
above. In the third enfibodiment, therefore, the 
switches 230 and 231 are turned on and the display 
"HD" is superimposed only when all of the three con- 
dilfons, (1) I Si I < Vsi. (2) Di s Vqi or Dj ^ V^, and 
(3) the switch 226 is OFF, are satisfied. That is. pho- 
tographing is done in the high-resolution niK>de only 
when AF is near an in-focus state, exposure is nearly 
appropriate, and there is no zoom command. There- 
fore, in the third embodiment an unnecessary pixel 
shift is not performed in an out-of-focus state, when 
the entire image is black or white on the screen, or 
during zooming. 

In the third embodinrient, the pixel shift and the 
superimpose display are performed when all of the 
above conditions (1 ) to (3) are met, and so the method 
can be considered to be most eff ident However, the 
pbcel shift and the superimpose display can also be 
done when one or two of the condittons (1) to (3) are 
met Even in this case, a power-saving effect can be 



obtained to some extent 

According to the third embodiment as discussed 
above, whether the pixel shift is to be performed is 
determined in accordance with the input signal to the 

5 CPU for controlling the image sensing apparatus. 
This prevents interference between the pixel shift 
and other automatic adjusting functions of the image 
sensing apparatus. In addition, an unnecessary pixel 
shift is not performed in an out-of-focus state or when 

io exposure is inadequate. This effectively saves the 
consumptton power. Consequently, it is possible to 
constantly obtain high-resolutton images with a high 
efficiency. 

is <4th Embodiments 

The fourth embodiment according to the present 
invention will be described below. 

Fig. 6 Is a block diagram showing the arrange- 

20 ment of an image sensing apparatus of the fourth em- 
t)odiment Reference numeral 301 denotes a record- 
ing system induding CCDs. A trichromatic separation 
prism 310 separates light from an object to be photo- 
graphed into three colors, red (R), green (G), and t>lue 

25 (B), and supplies these three light components to 
three CCDs 311a to 311c. R, G, and B filters are ad- 
hered to the CCDs 311a, 311b, and 311c, respective- 
ly. In the fourth embodiment, the pixel arTangen>ents 
in the three CCDs 31 1 a to 31 1 c are precisely kjentk»l. 

30 That is, no pbcel shift is done between the CCDs, so 
an image of an object to be photographed is projected 
at the same position in each CCD. Reference numeral 
312 denotes an A/D converter for converting outputs 
from the CCDs 311a to 311c into digital signals; and 

35 3 1 3, a memory. The RGB image data obtained by the 
CCDs 311a to 311c are recorded in a recording unit 
314, such as a semiconductor memory card or a hard 
disk, via the A/D converter 312 and the memory 313. 
This recording operatk)n is oontrdled by a CPU 31 5. 

40 An image sensing system 302 controls exposure, 
focus detectk>n, zooming, and the like processing. A 
VAP 303 is a pixel shifting means for changing the rel- 
ative positkMi of an image of an object with respect to 
each CCD. As explained In Japanese Patent l^id- 

45 Open No. 2-124518. the VAP 303 is an optical ele- 
ment formed by sealing a transparent elastic body be- 
tween parallel plates such as glass plates. The optical 
performance of a transmitted light beam is arbitrarily 
changed by changing the angles of these parallel 

50 plates using an external biasing force. Examples of 
the transparent elastic body sandwiched between the 
parallel plates are water, alcohol, glycol, silkxine oilr 
silicone gel, silicone rubber, and organk: oil. 

A motion detector 331 detects the amount of a 

55 camera shake in real time by using an acceleratk>n 
sensor, in accordance with the detected anrKxint, the 
front and rear glass plates of the VAP 303 are moved 
by a driving mechanism (not shown), thereby chang- 
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ing the vertical angle of the VAP 303. In this manner, 
a blur of an Image caused by the camera shake Is cor- 
rected. A drive controller 330 performs drive control, 
such as focus detection and exposure of the Image 
sensing system 302, and drive control of the driving 5 
mechanism for the glass plates of the VAP 303. The 
drive timing of the drive controller 330 is controlled by 
the CPU 315. 

A high-frequency component detector 304 ex- 
tracts a high-frequency component from data in a io 
predetermined area of the memory 313. The CPU 315 
and the drive controller 330 keep driving the image 
sensing system 302 until the high-frequency compo- 
nidnt is maximized, i.e., an in-focus state is obtained. 
The focus detectk)n method in the fourth embodiment is 
is a so-called "hill-dimbing servo method" which is 
well-known in the field of a vkJeo camera and in which 
an image sensing optical system is so controlled that 
a high-frequency component of an output signal from 
a CCD is maximized. This niethod Is disclosed in Ish- ~20 
Ma et al., 'Automatic Focus Adjustment of Televiston 
Camera According to Hill-dimbing Servo Method", 
NHK Technotogy, Vol. 1 7, No. 1 , and so a detailed de- 
scriptk>n thereof will be omitted. 

In the fourth embodiment, a pbcel shift is per- 25 
formed by changing the relative position of an image 
of an object with respect to each CCD by further 
slightly changing the vertical angle of the VAP 303. If 
the image sensing apparatus vibrates during photo- 
graphing, the Influence of the vibratton can be re- 3o 
duced by superposing the driving of the pbcel shift on 
the driving of the VAP 303 for vibration isolation. Fig. 
9 is a timing chart showing the state in whteh signals 
exposed and stored in the CCDs are sequentially 
read out while the VAP 303 is performing the pixel 35 
shift Since the operations of the CCDs 311a to 311c 
are exactly the same, the operatbn of tiie CCD 31 lb 
will be described as an example. 

Referring to Fig. 9, A1 represents the first expos- 
ure, and the read of a signal stored by the exposure 40 
A1 is represented by CI . That is, after the exposure 
A1 is completed, the read CI of the stored signal is 
started, and at the same time the VAP 303 Is driven 
to perform a pbcel shift Likewise, signals stored in ex- 
posures A2, A3, and A4 are read out in C2, C3, and 45 
C4, respecth^ely. After each of the exposures A2, A3, 
and A4 is completed, the VAP 303 is driven to perform 
a pbcel shift 

The pbcel shift in the fourth embodiment will be 
described below. Fig. 10 shows the way the position so 
of an tn^ge of an object to be photographed is shifted 
on the CCD by the VAP 303. Reference symbols la, 
1 b, 1c, 2a, 2b, and 2c denote some pbcels on the CCD 
311b; and £, a pbcel pitch. In the fourth embodiment. 
It is assumed that the pixel pitches are identical in the ss 
vertical and horizontal dlrectk>ns. Note that since the 
pixel arrangennents of the CCDs 311a and 311c are 
identical with that of the CCD 311b, only the CCD 



311b will be described l>elow. 

After an image of an object is photographed once 
at the initial position, the VAP 303 is driven such that 
images of the object projected on the pixels la, lb, 
1 c, 2a, 2b, and 2c are shifted by a p/2 pitch to the right 
in Fig. 10. Thereafter, photographing Is executed 
again. The driving amount at this time is calculated as 
follows. 

For the simplicity of explanation, the positional 
relationship between the VAP 303 and the image 
sensing system 302 is extracted from Fig. 8 and Illu- 
strated In Fig. 11. In Fig. 11, symbols have the follow- 
ing meanings. 

6: The angle at which light rays are inclined by 
the VAP 303. 

a: The distance from the VAP 303 to an image 
pick up plane 352. 

d: The shift arrK>unt (pbcel shift anxxint) of an 
Image of an object on the image sensing plane 352. 
In this case, since d is suff k:lently small, the following 
equation is established: 

tend = d/a 

Since a » d, 6 d/a is estat)lished. 

In order for the pixel shift amount d to be p/2, 
therefore, it is only necessary to drive the VAP 303 un- 
til the inclination angle 6, with which 6 = d/a = p/2a Is 
established, is obtained. Generally, control of the 
VAP 303 is done by the angle defined between the 
front and rear glass plates constituting the VAP 303, 
i.e., the vertical angle, rather than by the indlnatfon 
angle 5 of light rays. Calculations for this control will 
be discussed in detail in the fifth embodln^nt to be 
described later. 

In Fig. 10, the images df the object are photo- 
graphed as they are shifted in units of p/2 downward 
and to the left in the same manner as mentioned 
above. Thereafter, the images are returned to the ini- 
tial positions. The data obtained by projecting colors 
R, G, and B while the object images are shifted four 
times in this way are rearranged in the memory 313. 
This makes it possible to reconstruct for each of R, 
G, and B, a high-resolution image having the number 
of pbcels whteh is four times the number of pbcels of 
each CCD. 

The high-frequency component detector 304 de- 
tects a high-frequency component from the images 
obtained as discussed above, e.g.. from an image of 
G component, thereby performing focus detectton. 
Since the pbcel shift Is done for the Image Information 
to be detected, the information amount is doubled In 
each of the vertical and horizontal directkMis. There- 
fore, a largeranKMjntof a high-frequency component 
is extracted than before the pbcel shift is perfonDed. 
The image sensing system 302 is kept driven until the 
high-frequency component is maximized, i.e., an in- 
focus state is obtained. 

According to the fourth embodiment as dis- 
cussed above, the image sensing apparatus includes 
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the means for photographing an image of an object to 
be photographed by using n (n is 2 or larger) Image 
sensors, and the means for exposing the Image of the 
object a plurality oF-number of times while changing 
the relative position of the image to each image sen- s 
sor. As a result, it is possible to obtain high-resolution 
images. This makes detection of a larger amount of a 
high-frequency component and therefore a more ac- 
curate focus detection feasible. 

10 

<5th Embodiment> 

The fifth emt)odiment of the present invention 
will be described below. 

An image sensing apparatus of the fifth embodi- is 
ment has an arrangement identical with that of the 
fourth embodiment shown in Fig. 8, so a detailed de- 
scription thereof will be omitted. In the fifth embodi- 
ment, of the three CCDs 311a to 311c illustrated In 
Fig. 8. the CCDs 311a and 311 b have a filter of G. and 20 
a pbcel shift is done between them. Note that the CCD 
311c has stripe filters of R and B. The arrangement 
of the fifth embodiment is similar to that disclosed in 
"Highvtsion 2/3-Rich compact CCD camera using dual 
green method" In Television Society Magazine, "Inv 25 
age infonmation Engineering and Broadcasting Tech- 
nology", Vol. 47, No. 2, 1993 described in "BACK- 
GROUND OF THE INVENTION". Also, a method of 
producing RGB signals for ail pixels from photograph- 
ed data is known to those skilled in the art Therefore, 30 
a method of a pixel shift with respect to a luminance 
signal of an image of an object to be photographed 
will be described below. 

Referring to Fig, 12, reference symbols 31a, 31b, 
31c, 32a, 32b, and 32c denote some pixels in the CCD 35 
311a. In the CCD 311b, on the other hand, pixels are 
arranged at posittons oonesponding to positk>ns be- 
tween the pixels in the CCD 311a. i.e., arranged at 
pixel-shifted positkms, as indicated by 31a', 31b*. 
31c', 32a', 32b', and 32c'. After photographing is per- 40 
formed in this state, the relative positions of images 
of an object to be photographed are shifted in the di- 
rectfon indk^ted by arrows In Fig. 12 by a VAP 303, 
and again photographing is performed. In this meth- 
od, a pbcel arrangement of a G component as a lumi- 45 
nance oompon^ Is originally shifted. Therefore, for 
a luminance component It is possible to obtain the 
number of pixels which Is equivalent to the number of 
pixels in the RGB three sensor method described in 
the fourth embodiment by performing photographing so 
a total of two times. 

Note that in the fifth embodiment, in two of the 
three CCDs the pixel shift Is done in both the horizon- 
tal and vertteal directions. However, the pbcel shift 
can also be performed only In one directfon. ss 

In addition, if n (2 ^ n) CCDs are employed and 
m (2 ^ m ^ n) CCDs having filters of the same color 
are pixel-shifted by 1/m each in the horizontal or vert- 



ical direction, ((k + 1) x m) pbcel data can be obtained 
for a luminance component by pbcel shifting an image 
of an object to be photographed to a position at which 
no pixel is present k times by using a VAP or the like. 
These data are rearranged in a memory 313. This 
makes it possible to reconstruct, for a luminance conrv- 
ponent, a high-resolutton image having a data 
amount (k + 1) times as large as the data amount of 
each CCD. 

According to the fifth embodiment as discussed 
above, it is possible to synthesize images with the 
same high-resolution as in the fourth embodiment 
within a shorter time than in the fourth embodiment 

<6th Embodiment> 

The sbcth embodiment according to the present 
invention will be described below. 

Conventfonally. a method has been proposed by 
whtoh an image of an object to be photographed is 
divided into a plurality of regions and photographed, 
and the photographed images are synthesized to ob- 
tain a high-resdutfon image. The sbcth embodiment is 
characterized by further increasing the resolutfon in 
this method. 

The arrangement of an image sensing apparatus 
of the sbcth embodiment is nearly identical with that 
of the fourth embodiment shown in Fig. 8 except that 
an optical system for forming an image of an object 
to be photographed on each CCD is different from 
that of the fourth embodiment Fig. 13 shows an opt- 
ical system in the sbcth emtxxliment, in whch the 
same reference numerals as in Fig. 8 denote the 
same parts and a detailed descriptfon thereof %tfill be 
omitted. 

Referring to Fig. 13, dotted lines 351 represent a 
light beam from an object to be ptiotographed. An im- 
age of the beam 351 is first formed on a primary inv 
age formatbn plane 352 via a VAP 303 and an image 
sensing system 302. The image of the object once 
formed on the primary image fonmatfon plane 352 is 
divided by a secondary image formation optical sys- 
tem 353 and formed as secondary images on CCDs 
354a and 354b. In this case the object images formed 
as the secondary images on the CCDs 354a and 
354b are essentially pbcel-shifted by the VAP 303. 
The images formed on the two CCDs 354a and 354b 
are synthesized in a memory 313 to yield an image 
having the number of pbcels whteh is twice the number 
of pbcels of one CCD. 

According to the sixth embodiment as discussed 
above, images with a higher resolution can be photo- 
graphed. 

In addition, in the sbcth embodiment a still higher 
resolution can be achieved by periodk^ally driving the 
VAP 303 and further performing a pbcel shift as illu- 
strated in Fig. 9 of the fourth emtxxliment, thereby 
performing photographing a plurality of times. 
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In each of the first to sixth embodiments descri- 
bed above, a VAP Is used as the pixel shifting means. 
However, the present invention is not limited to these 
embodiments and can be realized by using the follow- 
ing means as well. Figs. 14A to 14E illustrate means 
for changing the relative position of an image of an ob- 
ject to be photographed with respect to a CCD, i.e., 
pixel shifting means, Including a VAP used In the 
above embodiments. 

Fig. 14A shows the VAP method discussed 
above. Assuming that the vertical angle Is ^, the In- 
clination angle of a light beam Is d, and the refractive 
Index of a VAP is n. 6 s (n - 1) x ^ is established. 

Fig. 14B shows a reflection argument method. 
When a mirror surface reflecting plate is Inclined by 
0, $ = 2 X 0 is establtshed for an inclination angle 5 of 
a light beam. Note that in the pUel shifting methods 
in Figs. 14A and 14B, ^ and e are controlled by per- 
forming calculations such that an Image of an object 
to be photographed is shifted a predetenmlned 
amount on an image sensing plane. 

Fig. 14C shows a rotary prism method. When a 
prism with a width L is inclined by ^, a light beam is 
shifted parallel by d which is given by d ((n - 1) x L 
X ^)/n. 

Fig. 14D illustrates a moving prism method. 
When two prisms each having a vertical angle a and 
a refractive index n are arranged in the reverse direc- 
tions as in Fig. 14D, the relationship between a dis- 
tance D between the prisms and a distance d by which 
a light beam is shifted is given by d ^ (n - 1) x a x D. 
Therefore, an Image can be shifted by changing D. In 
the pbcel shifting methods iOustrated in Figs. 14C and 
14D. (pixel shift amount) = d when the prism or the 
prisms are arranged imnrvediately before the CCD. If 
another optical system is inserted between the prism 
or the prisms and the CCD, the driving amount d need 
only be determined by taldng account of the effect of 
that optical system. 

Fig. 14E illustrates a moving lens group method. 
A lens group with a focal length f arranged in a taking 
lens group system is vertically shifted by D with re- 
spect to the optical axis. Assuming the distance be- 
tween the lens group and the in-focus plane is I, a 
shifted amountd of an image on the in-focus plane Is 
represented by d & I x D/T. 

As discussed above, the VAP method, Fig. 14A, 
as explained In the fifth embodiment is merely one 
example of the pixel shifting methods, so the means 
of dynamically shifting pixels can be achieved by va- 
rious methods. 

Anrx)ng other methods by which similar effects 
can be obtained are a method as disclosed in Japa- 
nese Patent Publication No. 57-31701 . in which a po- 
larizer capable controlling the direction of polarized 
light is anranged In an optical path; a method as dis- 
closed in Japanese Patent Laid-Open No. 3-201883, 
in which an entire image sensing apparatus is dis- 



placed; a method in which an Image is obtained by 
displacing an Image sensing optical system either 
partially or entirely to change the relath^e positional 
relationship between the image formed by an Input 

5 optica) system and a light-receiving portion of a CCD 
with time; and a method in which a CCD itself is vh 
bra ted on a plane normal to the optical axis by using 
a piezoelectric element or the like. 

As many apparently widely different embodi- 

10 ments of the present invention can be made without 
departing from the spirit and scope thereof. It Is to be 
understood that the Invention Is not limited to the spe- 
cific embodiments thereof except as defined in the 
appended dainr^. 

IS 

Claims 

1 . An Image sensing apparatus comprising: 

20 image forming means for forming an Im- 

age of an object to be photographed; 

image sensing means for photographing 
the image of the object formed by sakJ image 
forming means; 

25 displacing means for displacing the image 

of the object formed by said image forming 
means and said image sensing means relative to 
each other; and 

control means for performing switching 

30 between a first image sensing mode in which 
said displacing means performs a vibratk>n cor- 
rection to yield a low resolution image and a sec- 
ond image sensing mode in which said displacing 
means performs a pbcel shift to yield a high- 

35 resolutton Image. 

2. The apparatus according to claim 1 , wherein 

said Image sensing means consists of a 
plurality of image sensors each conespondtng to 

40 one pixel, and 

in the second Image sensing mode the inv 
age of the object is shifted by 1/N (N Is a natural 
number of not less than 2) of a pbcel pitch of saM 
image sensing means by said displacing means, 

45 and resulting N images of the object are photo- 
graphed and synthesized to yield a high- 
resolution image. 

3. The apparatus according to daim 2, further conv 
50 prising a memory tor synthesizing the N Images 

of the object obtained in the second image sens- 
ing mode. 

4. The apparatus according to claim 2, further conv 
55 prising vibration detecting means for detecting a 

vibration cyfsakJ Image sensing apparatus, 

wherein In the first Image sensing mode 
said displacing means is controlled in accor- 
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dance with an amount of the vibration detected by 
said vibration detecting means. 

5. The apparatus according to daim 1 , further com- 
prising motion detecting means for detecting a 
motion vector of the object, 

wherein said control means performs 
switching between the first and second image 
sensing modes in accordance with information on 
the motion vector of the object detected by said 
motion detecting means. 

6. The apparatus according to daim 1 , wherein said 
displadng means indudes a variable angle prism 
and an actuator for controlling a vertical angle of 
said variable angle prism. 

7. The apparatus according to daim 2, wherein said 
Image sensor is a CCD. 

8. The apparatus according to daim 1 , wherein said 
displacing means changes an argument of a mir- 
ror surface reflecting plato. 

9. The apparatus according to daim 1 , wherein said 
displadng means rotates a prism. 

10. The apparatus according to daim 1, wherein said 
displacing means changes relative positions of a 
plurality of prisms. 

11. The apparatus according to daim 1 , wherein said 
displadng means changes a position of a lens 
group with respect to an optical axis. 

12. An image sensing apparatus comprising: 

image forming means for forming an inv 
age of an object to be photographed; 

image sensing means for photographing 
the image of the object fbnfned by said image 
forming means; 

displadng means lor displadng the image 
of the object formed by said image forming 
means and said Image sensing means relative to 
each other; 

synthesizing means for obtaining a high- 
resolution image by synthesizing a plurality of im- 
ages of the object obtained by performing a pixel 
shift by said displadng means; 

changing means for changing photograph- 
ing conditions; and 

control means for determining, in accor- 
dance with the photographing conditions 
changed by said changing means, whether said 
displacing means is to be driven. 

13. The apparatus according to daim 12, wherein 

said changing means Indudes automatic 



focusing means for obtaining an in-focus state by 
automatically controlling said image forming 
means in accordance with the Image of the ob- 
ject and 

5 said control n^ans drives said displadng 

means if the in-focus state is obtained by said au- 
tomatic focusing means and does not drive said 
displadng means if no In-focus state Is obtained. 

10 14. The apparatus according to daim 12, wherein 
said changing means indudes automatic 
exposing means for obtaining a proper exposure 
by automaticaily controlling said inrage fomrring 
means in accordance with the image of the ob- 

IS ject, and 

said control weans drives said displadng 
means if the proper exposure is obtained by said 
automatic exposing means and does not drive 
said displacing means if no proper exposure is 

20 obtained. 

15. The apparatus according to daim 12, wherein 

said changing means indudes zooming 
means for oont rolling a photographing magnif icah 
25 tion of the Image of the object, and 

said control means drives said displadng 
means if said zooming means is not being driven 
and does not drive said displadng means If said 
zooming means is being driven. 

30 

16. The apparatus according to daim 12, wherein 
said displadng means indudes a variable angle 
prism and an actuator for controlling a vertical an- 
gle of said variable angle prism. 

35 

17. The apparatus according to daim 12, further 
comprising viewing means through which the inv 
age of the object can be viewed, 

wherein when said control means drives 
40 said displadng means, information indicating the 
driving is sent to said viewing means. 

18. The apparatus according to daim 12, wherein 
said displacing means changes an argument of a 

45 minor surface reflecting plate. 

19. The apparatus according to daim 12, wherein 
said displadng means rotates a prism. 

50 20. The apparatus according to daim 12. wherein 
said displadng means changes relath^e positions 
of a plurality of prisms. 

21. The apparatus according to daim 12, wherein 
ss said displacing means changes a position of a 

lens group with respect to an optical axis. 

22. An image sensing apparatus comprising: 

14 
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image forming means for forming an im- 
age of an object to be photographed; 

image sensing means for photographing 
the image of the object formed by said image 
forming means; 5 

displacing means for displacing the Image 
of the object formed by said image forming 
means and said image sensing means relative to 
each other; 

synthesizing means for obtaining a high- io 
resolution image by synthesizing a plurality of inv 
ages of the object obtained by performing a pixel 
shift by said displacing means; and 

vibration correcting means for selectively 
performing a vibration correction by using said is 
displacing means. 



said displacing means rotates a prism. 

29. The apparatus according to claim 22, wherein 
said displacing means changes relative positions 
of a plurality of prisms. 

30. The apparatus according to claim 22, wherein 
said displacing means changes a position of a 
lens group with respect to an optical axis. 

31. An imaging apparatus comprising means for 
forming an image of an object, means for sensing 
the image and means for effecting relative move- 
ment between the forming and sensing means to 
enable motion con^>ensation and/or to enable 
formation of a high resolution sensed image. 



23. The apparatus according to daim 22, wherein 
said image sensing means consists of a 
plurality of image sensora each oonesponding to ~ 20 
one pbcel, 

said displacing means displaces the inv 
age of the object by 1/N (N is a natural number of 
not less than 2) of a pbcel pitch of said image 
sensing means, and 2S 

sakj synthesizing means synthesizes re- 
sulting N images of the object to yield a high- 
resolution image. 



32. An imaging apparatus according to claim 31. 
which further comprises means for combining a 
number of images sensed by the sensing means. 



24. The apparatus according to daim 23, wherein 30 
said image forming means separates the 

image of the object to form a plurality of images, 
the same number of said image sensing 

means as the number of the images formed by 

said image forming means are provided, and 35 
In at least two of said plurality of image 

sensing means arrangements of imajge sensora 

are pbcel-shifted from each other. 



25. The apparatus according to daim 22, wherein 40 
said image forming means forms at least 

two secondary images of the object, 

said displadng means displaces the at 

least two secondary images relative to each 

other, and 45 
said image sensing means photographs 

the at least two secondary images. 



26. The apparatus according to daim 22, wherein 
said displadng means indudes a variable angle so 
prism and an actuator for controlling a vertical an- 
gle of said variable angle prism. 



27. The apparatus according to daim 22, wherein 
said displacing means changes an argument of a 55 
mirror surface reflecting plate. 



28. The apparatus according to daim 22, wherein 
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